
National Aeronautics and Space Administration 


Assuring excellence in safety and mission critical software since 1993 


Independent Verification & Validation Program 





IV&V Program 



• Established 1993 

• Staff of ~250 personnel 

— ~45 Civil Servants 
— TASC contractors (prime) 

— Other support contractors 

• ISO - 9001 Certified 
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IV&V Program Services 



• Life-cycle IV&V 

• Independent Assessments 

• Safety & Mission Assurance Support 

- Common support infrastructure for assuring core 
Software Assurance functions across the Agency 

- Cybersecurity & Information Assurance 

- Secure Coding 

• Software Assurance Research Program (SARP) 

• Jon McBride Software Testing And Research 
(JSTAR) Laboratory 

- Independent Test Capability (ITC) 

- Robotics 

- Cybersecurity Lab 

• STEM Outreach 






Sample of IV&V’s History 




• All High Priority NASA Missions: 

- Human Exploration - SLS, MPCV, GSDO, Shuttle, Station 

- Commercial Crew - Boeing, SpaceX 

- James Webb Space Telescope 

- MAVEN (in the news currently, just reached performed Mars Deep Dip) 

- Many more .... 

• FAA 

- Independent assessment of a contract management system 

• New York City 911 - ECTP 

- Systems Engineering IV&V effort worked in collaboration with NYC Mayors office to 
provide assurance that the emergency communications program 

• Jet Propulsion Laboratory 

- Performing a Blue Team Vulnerability Assessment against mission threads 

• Department of Energy 

- Drafting agreement for independent assessment related to information assurance 

• Department of Veterans Affairs 

- Currently drafting agreements to perform systems, software, and security IV&V activities 
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JSTAR & 

Independent Test Capability Team 


A 




S Create specialized group for 
simulation and test 

ITC team formed in 2009 
ITC Charter 


The NASA IV&V Program's Independent Test 
Capability (ITC) is responsible for acquiring, 
developing and maintaining adaptable test 
environments. These test environments enable the 
NASA's IV&V Program to perform dynamic analysis 
of software behaviors for multiple NASA missions. 

The ITC team is the expert in simulation and the 
IV&V project teams are experts in the systems. 



IV&V project teams utilize the solutions developed 
by the ITC team within the JSTAR laboratory. 



s Provide infrastructure to support efforts 

JSTAR Lab Built in 2010 


nsTAts - 

Jon McBride Software Testing 
ft Research Lab 


NASA IVftV 
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■f IV&V Projects Utilized JSTAR 

- Autonomous Flight Safety System (AFSS) 

- Global Precipitation Measurement (GPM) 

- International Space Station (ISS) 

- James Webb Space Telescope (JWST) 

- Juno 

- Mars Atmosphere and Volatile Evolution (MAVEN) 

- Multi-Purpose Crew Vehicle (MPCV) 

- Space Launch Systems (SLS) 


NASA’s IV&V Program 
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ITC and JSTAR 


• Independent Test Capability (ITC) team was 
formed in 2009 


- Develop, maintain, and operate adaptable test 

environments for the IV&V Program that enables the 
dynamic analysis of software behaviors for multiple 
NASA missions 


• These capabilities are deployed within the JSTAR Lab 

— ITC team resides within JSTAR group 

— JSTAR Lab currently spans two physical rooms 


CD 


[B2-215 and B1 -226] 


ipt |wo 

Projects!!! 
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SWAT ■ TQ&E 


• Houses tools, provides area for capability development 
(Robotics, R&D, OC-Flight-1, etc.), and resources for 
performing dynamic analysis 

• Resources are managed by the JSTAR Lab Manager 
(Brandon Bailey) 
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JSTAR Layout 
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VMs running on JSTAR servers 
can be accessed from any 
machine or via wireless. 
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Network Configuration 
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NASA Corporate Wireless [NASA^ Intern at Access) 
I 1 JSTAR/JTC ]No Internet Access) 

[if 7 , 

A J5TAR/ITC Wireless (NolnternEt Access) 

- - ■ VCIaud Acce&S to Virtual Machines 
A Guest Wireless (Comcast Internet Access) 



Unique implementation 
of shared file storage 
with two non-bridgeable 
interfaces. 


Unique usage of the 
vCloud product allowing 
for external access to 
isolated lab. 


The JSTAR Lab is 
NOT connected to 
IVV network. 

All computers in the 
lab are connected on 
an internal JSTAR 
network 


NASA IV&V JSTAR 
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Work Performed in JSTAR 



• Jon McBride Software Testing And Research 
(JSTAR) Laboratory 

- Independent Test Capability (ITC) 

• Acquire, develop and maintain adaptable test environments 

- Robotics 

• Mostly small scale research oriented to train IV&V practitioner 

- Cybersecurity Lab 

• Mission Vulnerability Assessments, Penetration Testing Training, etc. 

- Cubesat Development 


NASA’s IV&V Program 
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Test System Approaches 




• The following four approaches have been utilized 
to provide test systems to the NASA IV&V Program 

1. Acquire a copy of a development project’s test system 

{MAVEN, MPCV, ISS, O-REx, GSDO, SLS*} 

2. Develop software-only test system in-house {jwst, gpm, 

JUNO, DSCOVR, InSight, SDLS Prototype} 

3. Setup a Hardware-in-the-Loop Test Environment {afss, 

JWST} 

4. Setup Remote Access and/or Physical Access to Test 
Systems (maven, smap> 


* SLS will have custom development in addition to provided simulation 


NASA’s IV&V Program 
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ITC Current Work 




• Projects Overview 

• International Space Station (ISS) 

• JWST Integrated Simulation and Test (JIST) 

• Multipurpose Crew Vehicle (MPCV) 

• Space Launch Systems (SLS) 

• Ground Systems Development and Operations (GSDO) 

• Osiris-Rex (O-REx) 

• NASA Operational Simulator [NOS) 

- Reusable Hardware Models 

- Custom Middleware with Interception Capability 

• System Test Automation 

• Hardware Modeling 

• Cube Sat 

• CCSDS SDLS Prototype Development/Testing 
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2012 Software of the 
Year Competition 


A 





GO-SIM Simulation Software Receives Honorable Mention at 2012 
Agency- Wide Software of the Year Competition 

High fidelity simulator with no hardware dependencies 

Simulation of operational spacecraft and ground system 

Controlled execution of simulation model and processor states (start, 

stop, pause, resume) 


Configurable on a single USB stick 


"GO-SIM provides a solution to reducing the 
cost of deploying Goddard Dynamic Simulator 
(GDS) to mission subsystem teams and test 
teams... GO-SIM also yields significant reuse 
potential for future in-house GSFC missions by 
incorporation of the GDS." 

Stephen Leake 

GSFC GDS Product Development Lead 


Architecture supports dynamic addition of external custom spacecraft 
components to simulation environment 

Fault injection of spacecraft hardware and measurements, software and 
environmental variables to test flight software management responses 

Step-wise execution of flight software in controlled, repeatable manner 

Execute mission scenarios and characterize timing behavior, memory 

usage, and CPU utilization 


NASA’s IV&V Program 
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Specialized Hardware 
in the JSTAR Lab 



• List maintained on JSTAR Wiki 

- BAE RAD750 6U (contains onboard spacwire and 1553] 

- BAE RAD750 3U (is in Chassis with 1553 and Spacewire cards] 

BK Precision Power Supply 

- Compact PCI (cPCI] Chassis 

- cPCI Bus Analyzer 

- Gespac 3750 (PowerPC 750] 

- FPGA Development Kits 

• One GR-CPCI-XC4V - http: //www.pender.ch/products cpci xc4v.shtml 

• Six GR-XC6S - http://www.pender.ch/products xc6s.shtml 

- Logic Analyzer (TLA6402 - http : / / www.tek.com /logic-analvzer / tla640 0 ] 

- MIL-STD-1553 Cards (ExpressCard and cPCI] 

- Oscilloscope (MSO4104B - http://www.tek.com/osdlloscope/mso4000-dpo400Q ] 

- PMC Carrier Card 

- Spacewire Test Set (SWTS] 


NASA’s IV&V Program 
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JSTAR's Virtualization 



Project X Analyst 


The lab utilizes server and 
desktop virtualization to 
improve the efficiency and 
availability of resources and 
tools. This provides the ability 
to run multiple virtual 
machines on each physical 
machine. Virtualization 
removes the physical server 
constraints and enables 
sharing of resources within 
the lab. 


JSTAR Servers* 


Project X 


Project Y 


ITC 
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Project X VM 1 


Project X VM 2 


Project X VM 3 


Project Y VM 1 


Project Y VM 2 


Project Y VM 3 


ITC VM 1 


ITC VM 2 


ITC VM 3 



* Currently have 9 ESXi server (mostly Dell R720s) but researching 
VMware's cloud offering. 
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Virtualization Features 




• Virtual Snapshots 

- Can take existing physical machine and convert to virtual machine 

• Easy backup of existing configurations 

• Only need Ethernet connection and administrator password 

- Can convert existing virtual machines (VirtualBox, Parallels, etc.) to VmWare 
format for use in the lab 

• Cloning/Templates 

- Once a working configuration is completed a clone or template can be created 
to spawn multiple copies for multiple users 

- Reduces time for configuration management/updating 

• Tool Testing 

- Can provide VMs (Windows, Linux, etc.) to any user to install, checkout, and 
test new tools 

• Single file delete for Lab Manager once work is complete and VM no longer needed 

• Can virtual separate networks to isolate traffic (Cybersecurity training) 
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Virtualization Benefits w/ 
Cyber Security 



• Due to their risky nature and potential impact 
to operational systems, cyber security training 
cannot be performed on live operational 
systems 

- Therefore, we developed a state-of-the-art training 
platform using virtualization that provides cyber 
training exercises using replicated vulnerable 
systems 

- We are also able to create "virtual" replicas of 
existing systems and perform training, 
reconnaissance, and testing against them in a 
controlled environment with the advantage of if the 
virtual system gets corrupted it can simply be 
reloaded with relative ease 

• We understand that textbooks and 
documentation only go so far, and are not able 
to provide the added benefit gained by actual 
hands-on experience 

- It is not feasible to offer all personnel their own 
replicated physical system and it is too risky to offer 
hands-on cyber training on operational networks; 
therefore we create each person their own virtual 
sandbox 
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Summary 



• IV&V has 20 year history providing software 
assurance to NASA 

— Also providing assurance to non-NASA entities 

• JSTAR is a multi-purpose lab 

— Simulation Development 
— HWIL Test Environments 

- Cybersecurity 

- Independent Testing 
— Training 
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Independent Test Capability 



Contact Information 

Presenter (s): 

Brandon Bailey 

brandon.t.bailey@nasa.gov 

304 - 629-8992 

iw-itc@lists.nasa.g ov 


IV&V Website: http://www.nasa.gov/centers/iw/home 
ITC Website: http : / /www.nasa.gov/centers /iw/ jstar /ITC.html 
JSTAR Website: http://www.nasa.gov/centers/iw/jstar/ISTAR.html 


